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Abstract. We prove a subelliptic estimate for systems of complex vec- 
tor fields under some assumptions that generalize the essential pseu- 
doconcavity for CR manifolds, that was first introduced by two of the 
Authors, and the Hormander's bracket condition for real vector fields. 
Applications are given to prove the hypoellipticity of first order systems 
P^ and second order partial differential operators. Finally we describe a 

class of compact homogeneous CR manifolds for which the distribution 
of (0, 1) vector fields satisfies a subelliptic estimate. 
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^ Introduction 

^ In this paper we prove a subelliptic estimate for sums of squares of com- 

plex vector fields. Namely, given a distribution 3(Af) of complex vector 
J, fields on an m dimensional real smooth manifold M, we find conditions for 

Cu the subellipticity of 3{M), i.e. the validity of the estimate 



(0.1) \\uf, < C \\u\\l + V \\L,{u)\\l Vn G C^{U), 




where f7 is a relatively compact open subset of M and e > 0, C > are 
positive constants, depending on U and on Li, . . . , L„ G 3(Af ), and C^{U) is 
the space of smooth complex valued functions on M, with compact support 
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contained in U . It is known that this estimate imphes the C°°-hypoenipticity 
in U of YJ"^^^ L*Lj (see e.g. MM)- 

Our result directly applies to the study of the overdetermined systems of 
first order partial differential operators on M, that are canonically associated 
to 3(Af) and to a C-linear connection on a complex vector bundle E — > M. 

We also investigate the C°°-hypoellipticity of more general second order 
partial differential operators on M, of the form 



(0.2) 



P{u) = y^ LjLj{u) + Lq{u) + au, 



where a is a complex valued smooth function in C^{M), Lo,Li, . . . ,Ln are 
smooth complex vector fields on M, and only the imaginary part of Lq is 
required to belong to the C°^(M)-linear span of Li, . . . , L„, Li, . . . , Z„. 

Our original motivation, and also our main applications, involve CR man- 
ifolds. However, the study of (0.1) and of the operators (0.2) is of inde- 
pendent interest, and related questions have been considered recently in 
[T8l [71 m [22l [23]. These papers show that the consideration of complex Lj's 
brings completely new phenomena, compared with the real case (see e.g. 

mi). 



Condition (1.21) for the subellipticity of ^{M) can be viewed as a gen- 
eralization, at the same time, of both the essential pseudoconcavity of |12] 



in CR geometry and Hormander's condition of [14J for the generalized Kol- 
mogorov equation. In fact, in ^we prove that, at points that are generic 



for ^{M) (in a sense made precise in Definition 2.15), it is equivalent to a 



condition (2.32), that involves semidefinite generalized Levi forms attached 



to 3{M) and their kernels. This quite explicit formulation was suggested 



by specific examples from fT. However, conditions (1.21) and (2.32) apply 



to more general contexts than CR geometry. At the beginning, we assume 
neither that 3(Af ) is a distribution of constant rank, nor that it satisfies any 
formal integrability condition, nor anything special about the intersection 
3{M)n3{M); but conditions of this type need to be imposed in ^to obtain 
the equivalence of ( |1.21 ) and (2.32). 

The work of |18| [3 [6] shows that the condition that the Lie algebra 
generated by ^{M) spans the full complexified tangent space is in general 
not sufficient either for subellipticity or for the C°°-hypoellipticity of the sum 
of squares. The present work is complementary, inasmuch as our sharpest 
results hold away from singularities and in the case where 3(Af) is a Lie 
algebra. 

We reduce the question of the subellipticity of 3{M) to one involving real 
vector fields. Indeed, our task is to find all real vector fields X that are 
enthralled by 3(M), i.e. satisfy 



(0.3) 



l^(^)lle-l<C 




Lj{u 




Vn G C^{U) 



for some Li, . . . ,Ln G 3{M) and e > 0, C > depending on X and U. In ^ 
we observe that this set is a module over the Lie algebra 2t3(M) generated 
by those real vector fields X for which (0.3) holds with e = 1. This was 
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esse ntially shown in ^6^. By an argument of J.J.Kohn (see |18j and Lemma 



1. 



below), we know that (ReZ) satisfies (0.3) with e = ^ for all Z G '^{M). 



On the other hand, on an open dense subset M' of M that excludes some 
singular points for 3(Af), one can check that a real vector field X satisfying 
(0.3) with e > ^ is necessarily the real part of some Z S 3(Af). Hence 



we conjecture that the real vector fields X, satisfying (0.3), coincide on a 
dense open subset of M with the elements of the 2t2(M)-module generated 
by the real parts of the elements of 3(Af )• In §2] we characterize, outside the 



singular set, the Lie algebra 2l3(M). Then equality ( |2.31 ), together with 



Lemma 1.8 would give a complete and explicit description of the set of real 
vector fields enthralled by 3(Af). 

A motivation for [12^ was to understand the structure of a large class of 
CR manifolds, of higher CR codimension, that are of finite type in the sense 
of [1] , and are not pseudoconcave in the sense of jTT] , because of the vanishing 
of their scalar Levi forms in some characteristic codirections. To prove 
also in this case a subelliptic estimate for the tangential Cauchy-Riemann 
operator on functions, two of the authors invented in [12j the notion of 
essential pseudoconcavity. Compared with the usual more restrictive notion 
of pseudoconcavity, it allows some of the scalar Levi forms to be zero. The 
subelliptic estimate is then obtained for weaker Sobolev norms than the 
2-norm in [11]. On essentially pseudoconcave almost CR manifolds the 
maximum modulus principle and the weak unique continuation principle for 
CR functions ( p^ll3j ) are valid. Under the additional assumption of formal 
integrability of the CR structure, it was also possible to prove, in case the 
CR manifold is either compact or real analytic, finiteness and vanishing 
theorems for the highest cohomology groups of the ^^f-complexes (|21j). 

Our new condition is more general than the weak pseudoconcavity of [T^] , 
as here we allow the scalar Levi forms in some characteristic codirections to 
be semidefinite. As a heuristic motivation, consider a CR submanifold M 
of a complex manifold F. If M is contained in a strictly pseudoconvex real 
hypersurface of F, then it is easy to find Lf^^ germs of CR functions that 
are not smooth. We can expect, vice versa, that all CR distributions on 
M are smooth when all germs of hypersurfaces through M in F are strictly 



pseudoconcave. This is indeed the case when our condition (2.32 ) is verified. 
It is discussed in f|5j 

As in [12 , the homogeneous examples have strongly inspired our inves- 
tigation. After considering general homogeneous CR manifolds in ^ in ^ 
we classify all minimal orbits of real forms G of G'^-homogeneous flag man- 



ifolds (see e.g. [Ml El [2]) that enjoy condition (1.21). In [T] §13], together 
with Prof. Medori, two of the authors gave the complete classification of the 
essentially pseudoconcave minimal orbits. Here we show that those satisfy- 



ing (1.21) form a strictly larger class of CR manifolds, on which the local 
CR distributions are smooth. 

We collected our results on hypoellipticity in Q Given a complex vector 
bundle E — > M, endowed with a C-linear connection V, for each com- 
plex vector field Z on M we obtain a differential operator V^, acting on 
the sections of E. We prove that the subellipticity of 3{M) implies the 



4 A. ALTOMANI, C. D. HILL, M. NACINOVICH, AND E. PORTEN 

C°°-hypoellipticity of {'^z)z<^3(M)- This yields, for a compact M, finite di- 
mensionality of the space of global solutions of {'^z{u))z£3{M) = Oi ^^d 
closedness of the range of C^z)ze5{M)- Tor CR manifolds, this implies that 
the cohomology groups H^' (M) of the tangential Cauchy-Riemann complex 
are Hausdorff. 

The subellipticity of 3(-^) gives C°°-hypoellipticity for the sum of squares 



and also for slightly more general operators (see Theorem 4.3 1 . The question 
of generalizing the Kolmogorov equation to the case of complex vector fields 
is more complicated. We obtained two different formulations in Theorems 



4.4 and |4.8[ In the former, we obtain C°°-hypoellipticity under the condition 
that the 2l3(M)-module generated by the real parts of the vector fields in 
3(M) and the "time" vector field span the whole tangent space at a point. 
This distinction between "space" and "time" vector fields makes this result 
weaker than the one of [13] for the case ^{M) = ^{M). This is the reason 



we prove in Theorem 4.8 a result generalizing [T^, under the condition that 



the real parts of the vector fields in 3(Af) satisfy (0.3) with e = 1 



1. SUBELLIPTIC SYSTEMS OF COMPLEX VECTOR FIELDS 

Let M be a smooth real manifold of dimension m. For U open in M, 
we denote by C°°{U) (resp. C^{U)) the space of complex valued smooth 
functions (resp. with compact support) in U, and by X{U) (resp. X'^(C/)) 
the Lie algebra of the smooth real (resp. complex) vector fields in U. 

Definition 1.1. A distribution of complex vector fields 3(Af) in M is any 
C°°(M)-left submodule of X^(M). This means that Zi + Z2 and fZ belong 
to 3(M) if Zi, Z2, Z £ 3(M) and / G C^iM). 
For each point p G M we set 

(1.1) ZpM = {Z{p) I Z G 3(M)} C T^M = C®w TpM. 

The dimension of the complex vector space ZpM is called the rank of 'b{M) 
at p. We do not assume in the definition that 'b{M) has a constant rank. 
The points where the rank of 'i{M) is not constant on a neighborhood are 
the singularities of 3(M). If 3(M) = {Z G X^{M) \ Z{p) G ZpM, Vp G M}, 
we say that 3(Af) has at most simple singularities. 
We say that '^{M) is formally integrahle if 

(1.2) [3(M),3(M)]C3(M). 

Distributions of real vector fields and their singular points are defined 
likewise. 

Definition 1.2. The distribution 'b{M) is said to be subelliptic at p G M if 
there is an open neighborhood C/ of p in M, a real e > 0, a constant C > 0, 



and a finite set Li,...,L„ of vector fields from 3(-^), such that (0.1 1 is 
satisfied, where || • \\^ and || • ||o are the e-Sobolev norms and the L'^-norm 
with respect to some Riemannian metric on M (see e.g. [S]). 

Remark 1.3. When 3{M) is the complexification of a distribution of real 
vector fields 2)(M) C X{M), then ( |0.1[ ), at a generic point p G M, is equiva- 
lent to the fact that 3(-M) and its higher order commutators span the whole 
complexified tangent space TpM (see, e.g. [HI [8]). However this condition 
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is neither necessary nor sufficient, and does not imply C°°-liypoellipticity of 
tlie associated sum of squares operators wlien tlie vector fields are complex 

(cf. HHEIEIES]). 

Definition 1.4. We say that 3(M) enthralls a vector field Z G X'^(M) if 
( V?7°P"'" (s Af, 3Li, . . . , L„ e 3(M), 3e > 0, C > s.t. 



(1.3) 



\Z{n)\\l^ <C{Y, \\H^)\\l + ll^llo 1 , Vn G C^{U). 



Set 

(1.4) S3(M) = {Z e X^(M) I 3(M) enthralls Z}. 

We notice that S;j{M) is a distribution of complex vector fields containing 
3(-Af), and S3(Af) n X(M) is a distribution of real vector fields, and that 
both are spaces of global sections of fine sheaves of left modules, the first 
over the sheaf of germs of complex valued smooth functions, the second over 
the sheaf of germs of real valued smooth functions on M. 

By the real Frobenius theorem we obtain (see e.g. |14j) 

Proposition 1.5. LetW;^{M) be the Lie subalgebra ofX{M) generated by 
3(M) + 3(-Af)- Then, if TQ^{M) is a distribution of constant rank on M, 
we have 

(1.5) §3(M) n X(M) C S3(M) C W^{M). 

Remark 1.6. The complex distribution ^{M) is subelliptic at p G M if and 
only if 

(1.6) {X{p) I X G S3(Af) n X{M)} = TpM. 

Remark 1.7. If Z G S;^{M) and A_i is a properly supported pseudodif- 
ferential operator of order (—1) having its symbol in the class 5'j~q, then 
A_i o Z is a subelliptic multiplier for 3(-Af) in the sense of J. J.Kohn (see e.g. 

HZl)- 

Pseudodifferential operators will be an important tool in the following. 
For their definition and properties we refer to \15\ Chapter XVIII]. 

If U is an open subset of M, and s a real number, we shall denote by 
^^{U) the space of properly supported pseudodifferential operators in U, 
of order less or equal than s, with symbol in Sf^. For each coordinate 
neighborhood V C U, a A €z ^*([/) is defined, in the local coordinates, by 

(1.7) A{u)= [[ e'^'^-y'^^a{x,Ouiy)dydC, for n G C^(y), 

with a GC°°(y xM"') and 

VK (s V, Va, /3 G N*", 3C = C{K, a,(3)>0 s.t. 



(1.8) 



1/3. 



D'^Dla{x,0 < C(l + \C\y-\"\, V(x,e) eKx 



The fact that A G ^*(C/) is properly supported means that for every K <£ U 
there is K' <£ U such that 

(1.9) supp(u) C K ^ supp(A(ti)) C K'. 



A. ALTOMANI, C. D. HILL, M. NACINOVICH, AND E. PORTEN 



The following Lemma is essentially contained in |18j p. 949]. 

Lemma 1.8. Let Z G X (M) be any complex vector field. For every rela- 
tively compact open subset U <^ M there is a constant C > such that 

(1.10) \\Ziu)\\\<C{\\Ziu)\\l + \\u\\l) yu€C^iU). 
Hence 

(1.11) 3(M)+3WcS3(M), 

(1.12) {z + z\z e 3(M)} C S3(M) n X(Af ). 

Proof. Let Z G 3(M) and [/"P'^'" (s M. Then, with Aq G ^°(C/) and con- 
stants Ci, C2 > 0, we obtain 

\\Z{u)f_, = iZiu)\Ao{u))o < \{Aliu)\Ziu))o\ + Ci||n||2 
2 

< C2 {\\Z{u)\\o + \\u\\o) ||n||o, Vu G C°°(C/). 
This yields ( |1.10D , and hence also ( |1.11[ ) and ( |1.12[ ). □ 

In particular, if 3(-/Vf ) / {0}, the distribution E;i,{M)r\X{M) of rea/ vector 
fields that are enthralled by 3(Af ) is not trivial. 

Definition 1.9. We denote by E^{M) the set of Z G X^(M) such that 
VC/°P^° (E M, 3Li, ...,Ln€ 3(M), 3C > s.t 



(1.13) 



\Z{u)\\l<C 



^\\Hu)\\l + \\nro 



VnGC^(C/). 



Also E3(M) is a distribution of complex vector fields, with 

(1.14) 3(M) C E3(M) C S3(M). 
As a consequence of Lemma |1.8[ we get 
Lemma 1.10. 

(1.15) E3(M)+%p?ycS3(M). 
Definition 1.11. Set 

(1.16) A3(M) = E3(M) n X(M), 

(1.17) 2t3(M) = the Lie subalgebra of X(M) generated by A^{M), 

(1.18) T^i'^(M) = {Z + Z| Zg3(M)}, 



.(h). 



(h-i), 



(1.19) T^"^(Af) = nA:,y] I a: G A3(M), Y G T^ ^(M)}\ , for /i > l. 



(1.20) T3(M) = ^T^ ^(M). 



/i=0 



We shall consider the condition at p G M: 

(1.21) {X{p) I X G T3(M)} = TpM. 



Remark 1.12. If condition (1.21) is satisfied at a point po G M, then it is 
also satisfied at all points p in an open neighborhood U of po- 
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It is convenient to introduce the notation [Zi, . . . , Zm] for the higher order 
commutator of smooth real or complex vector fields. It is recursively defined 

by 

'[Zi] = Z,, 
(1.22) l[Zi,Z2] = ZiZ2-Z2Zi, 

^ [Zi, Z2,..., Zm] = [Zi, [Z2, ..., Zm]] for m > 3. 

Proposition 1.13. The distribution of real vector fields 1.:^{M) is an 2l3(M)- 
Lie-module. 

Proof. We prove by recurrence on r > 1 that, if Xi, . . . , X^ G A3(M) and 
Y e T3(M), then also [[Xi, . . . ,Xr],Y] £ l^iM). This follows from the 
definition of T3(M) for r = 1. Assume now that r > 1 and that T3(M) 
contains all commutators [[Xi, . . . , Xj._i], Y] with Xi, . . . , X^-i G K^{M) 
and Y £ 1^{M). U Xi,...,Xr £ A^iM) and Y £ l-^iM), we obtain 

[[Xi, ...,Xr],Y] = -[[X2, ...,Xr], [Xi,Y]] + [Xi, [[X2, ...,Xr],Y]]. 

Since [Xi,y] £ T3(M), by our inductive assumption the first summand on 
the right hand side also belongs to T3(M). By the inductive assumption, 
the commutator [[X2, . . . ,Xr],y] belongs to T3(M), and hence also the 
second summand in the right hand side belongs to %-^{M). The proof is 
complete. D 

Proposition 1.14. The Lie algebra of real vector fields 2l3(M) is contained 
mS3(A/)nX(M), and8^{M)nX{M) is an^^{M)-Lie-submodule ofX{M). 
In particular, we have the inclusion 



(1.23) 



13(M) cS3(Af)nX(M). 



Proof. Let U he, a, relatively compact open subset of M. Assume that X £ 
A3(M), Y £ §3(M) n X(M) and let e > be such that that Y = Z satisfies 



the estimate in (1.3). We can assume that < e < j- If t/' is an open 



relatively compact subset of U , we have, with some Ae_i £ '^'^ (U) and 
suitable positive constants Co, Ci, 

||[X,y](n)|||_i<|([X,y](n)|A, 
< \{XY{u)\K 

<\{KU{Y{u))\X{u)),\ + \{X{u)\K 
+ Coh|| (||n||o + ||y(n)||e-: 



^i(^i))o| 
_i(n))o| + |(yX(n)|A, 



-i(^^))o| 
-i^(^))l 



+ 



<Ci 



l^(")llo) 

yu £ c°°{u') 



\Xin)\\l + \\Y{u)\\U + \\u\\l), 
The last term of this chain of inequalities is bounded by a constant times 
(X^?=i ll-^j(^)lP + ll'^lloj' ^°^ ^ suitable choice of Li, . . . ,L„ £ 3{M). This 
shows that [X, Y] £ 8^{M) n X{M). 

Since A3(M) C S^{M) n X(M), also 2l3(M) C §3(1/) n X(M). The 
argument in the proof of Proposition 1.13 shows that, since [A3(M), S3(M)n 
X(M)] C S3(M) n X(M), this distribution is an 2t3(M)-Lie-submodule of 
X(M). Then the inclusion (1.23) is a consequence of the inclusion (1.12). D 



By using Proposition 1.14 we obtain 
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Corollary 1.15. Let ^{AI) be a smooth distribution of complex vector fields 
on M. Then 3(M) is subelliptic at all points p £ M at which condition 



(1.21) of Definition 1.11 is satisfied. 



Corollary 1.15 is a trivial consequence of the inclusion ( 1.23 ). However, in 
q2]we will show that actually we are able, in case 3(Af ) is formally integrable, 
to compute explicitly the left hand side of ( 1.21[ ) at the points of an open 
dense subset of M, where 3(-M^) satisfies some genericity assumptions. 

Remark 1.16. When '^{M) is the complexification of a distribution of real 
vector fields, then %-^{M) = T3(M), and condition (1.21) is equivalent to 
Hormander's condition in [T3|. Thus Corollary 1.15 can be viewed as a 



generalization of the analogous result for distributions of real vector fields. 

Remark 1.17. li'^{M) is the distribution of (0, l)-vector fields of an almost 
CR manifold M, it follows from ^below that the essential pseudoconcavity 
condition of HI] imphes that E^{M) = 3(M) + 3(M ) and that M is of 



finite type in the sense of [3j. Therefore, Corollary 1.15| also generalizes |12| 
Theorem 4.1]. 



2. The distributions K-7,{M) and Q-^i^M) 



As pointed out after the statement of Corollary 1.15, condition (1.21) 



becomes an effective criterion for subellipticity when it is possible to give an 
explicit description of K^{M), or of some nontrivial part of it. We begin by 
giving an upper bound for E3(M). 

Lemma 2.1. Let 3(-^) be a distribution of complex vector fields. 

If (3(M) + ^(M)) has at most simple singularities, and in particular if 
{^{M) -\- ^{M)) has constant rank, then 



(2.1) 



E3(M)C3(M)+3(M). 



Proof. Since K^(M) C E, , ■^(M), we can reduce the proof to the case where 
3(M) = 3(M) +3(-/Vf ) is the complexification of a distribution of real vector 
fields and Z G E3(M) is real. Fix p E M and take a coordinate patch 
[/ of p in M, centered at p, for which (1.13) holds, for real Li, . . . ,L„ S 



X{M) n 3(M) such that Li(p), . . . , L„(p) generate ZpM. We apply the 



inequality in (1.13) to the test function Ut{x) = r 4 x{x)e^ 

where x(^) ^ ^^(C/) satisfies x{x) = 1 foi' 2; in a neighborhood of 0. Denote 

by z{x,^) and £j{x,^) the symbols of Z and Lj, respectively. We obtain 



Lj{ur) = T 4 (r£j(x, i + ix) + Lj{x)) e 
Computing the integral by the change of variables y = x 



ir(x,0-iT/2)\x\\ 

T, we obtain 



|L,-(ut 



xHy/V^)\ej{y/V^,iC + y/V^)\'e-\y\'dy + o{T-^). 



Likewise, we have 



z{nr)fo= I xHy/V^My/V^,i^ + y/V^)\'e-\y\'dy + o{T~^) 
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By letting r tend to cxd in the estimate (1.13), we obtain that 

n 
3=1 

Since Li,. . . ,Ln are real, the above inequality implies that Z{p) 6 ZpM. 
Since p was an arbitrary point of M, this implies that Z £ ^{M). D 



Remark 2.2. The proof of Lemma 2.1 yields a stronger statement: 

Let 5{M) be a distribution of complex vector fields and assume that 
3(M) + 3(-A^) has at most simple singularities. If Z £ S^{M) and 



(2.2) 



VC/°P^'^ (EM, 3e > -, 3Li, . . . , L„ E 3(M), 3C7 > s.t. 



\Ziu)\\l,<c{Y,\\Lj{n)\\l + \\u\\ 



Vm G C^{U), 



thenZ e5{M) + :5{M). 

It suffic es in deed to apply (2.2), in a coordinate patch U, as in the proof 
of Lemma 2.1 , to the test functions v-r = t^^~^ ^'"^Ut, with 7^ < e' < e, and let 
r -^ +C!0. Then, if ij{0, ^) = for j = 1, . . . ,n, the right hand side of (2.2 ) 
stays bounded, while the left hand side tends to +00, unless z{p,S,) = 0. 



2.1. The distribution 03(M). Lemma 2.1 suggests that, in order to find 
non trivial elements of E3(M), one should search in 3(M). To this aim, we 
introduce the following 



Definition 2.3. Given a distribution 3(-M) of complex vector fields, we set 

(2.3) e^{M) = \ze 3(M) 



3r > 0, 3Zi, ...,Zre 3(M), s^L 1 

[Z, Z] + E,'=i[^., Z,] G 3(M) + J(M)j ■ 

Lemma 2.4. The set Q^{M) is a distribution of complex vector fields. 

Proof. Clearly, if Z E Q},{M) and </> E £{M), the product (f) Z also be- 
longs to 03(M). To prove that 03(M) contains the finite sums of its ele- 
ments, it suffices to show that, if, for some r > 1, Zq, . . . , Z^ E Q-^{M) and 
TJ'j=o[Zj,Zj] E 3(M) +3(M), then also Zq + Zi E e3(M). This follows 
from: [Zq + Zi, Zq + Zi] + [Zo-Zi, Zq - Zi] = 2 ([Zq, Zq] + [Zi, Zi]) . D 

Lemma 2.5. Let 3(Af) 6e a distribution of complex vector fields and let 
<S)^{M) be defined by ^^. Then 

(2.4) e^p^cE3(M). 

In particular, if Z e Q^{M), then Z + Z e A^{M). 

Proof The proof closely follows that of [HI Theorem 2.5]. If Z E Q^iM), 
by (2.3), there are Zi, . . . , Z,., Zr+i E 3(-M) such that 



(2.5) 



[Z, Z] -I- 2_^[Zj, Zj] — Zr+l — Zr+l- 
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Set Zq = Z and let Z* = —Zj + aj, with aj G C^iM)^ the l? formal adjoint 
of Zj, for < j < r + 1. Integrating by parts, and using (2.5) to compute 
the sum of the commutators, we obtain, for all u £ Cq°(M), 

r r 

^||Z,(u)||g = -Y,{{Zj--aj){Zj{u))\u)o 

j=0 j=0 

r 

= -^{{[Zj,Zj]{u)\u)o + {ZjZj{u)\u)o - {Zj{u)\aju)o} 

J=0 

r r+l 

= Y^ \\Zj{u)\\l + ^Re(Zj(n)|6jn) +Re(n|6'u)o, 
i=o j=o 



where bj,b' £ C°°{M). Hence the inequality in ( 1.13| ) (with Z replacing Z) 
follows, with n = r + 2 and Lj = Zj^i for j = 1, . . . , r + 2. □ 

2.2. The characteristic bundle and the scalar Levi forms. Next we 

define the characteristic bundle of 3{M) and the analogues, for a general 
distribution '^{M), of the scalar Levi forms of CR manifolds. 

Definition 2.6. The characteristic bundle of 3(M) is the set H^M C T*M, 
consisting of the real covectors ^ with {L,^) = for all L € 3(Af)- 

If the set HpM of characteristic covectors at p E M is {0}, we say that 
3(M) is elliptic at p. 

For each p e M, the set H^M = H^M n T*M is a vector space. Its 
dimension diuiK^/pM is an upper semicontinuous function of p G M. In 
particular, if 3(Af ) is elliptic at a point po ^ ^j it is elliptic for p in an open 



neighborhood U of pQ. In this case (0.1) is valid with e = 1 by Garding's 



inequality. Hence obstructions to the validity of the subelliptic estimate 



(0.1 ) may come only from the characteristic codirections of '^{M). 



We restate condition (1.21 ) in terms of the characteristic bundle. 



rO; 



Proposition 2.7. Condition (1.21) at p £ M is equivalent to 
(2.6) 



V^ G if^M, with ^ / 0, 

3Zo G 3(M), Zi, . . . , Z, G E3(M) n %(M), 

S.t. i^{[Zi,...,Zr,Zo])^0, 



Proof. This follows because the elements of T3(M) can be expressed as linear 
combinations of the real parts of the [Zi, . . . , Z^, ZqJ's, with Zq G ^^M) and 
Zi, . . . ,Zr G IE3(M) n K^{M), and vice versa, the real and imaginary parts 
of these [Zi,...,Zr, Zq] belong to l^iM). D 

Definition 2.8. If ^ G H^M, we define the scalar Levi form of 3(-W') at £, 
as the Hermitian symmetric form 

(2.7) £i:{Li,L2)=iC{[Li,L2]) for Li,L2G3(M). 

The value of the right hand side of ( |2.7| ) only depends on the values Li{p), 
L2{p) of the two vector fields Li, L2 at the base point p = '/r(^). Thus ( |2.7[ ) 
is a Hermitian symmetric form on the finite dimensional complex vector 
space ZpM. 
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Remark 2.9. In the case where 3{M) is the space of (0, l)-vector fields of an 
abstract CR manifold, it was shown in |11] that the subelliptic estimate ( |0.1[ ) 
is valid with e = 1/2 under the assumption that, for every ^ E HpM\{0}, the 
Levi form £^ is indefinite; this assumption was weakened to allow £^ = 
for some nonzero characteristics £, in [12]. These results suggest that the 



obstructions to the validity of (0.1) come from the characteristic ^'s for 
which £^ 7^ is semidefinite. 

2.3. The distribution ]K3(M). Our next aim is to relate K;^{M) and the 
Levi forms associated to 3(M). 

Definition 2.10. Define 

(2.8) H®M = {e G H°M | £5 > O} , 

(2.9) K3(M) = {Z G 3(M) I C^{Z, Z) = 0, V^ G H®M}. 

Proposition 2.11. For every distribution 5{M) C X^{M), the set K-7,{M) 
is also a distribution. 

Assume in addition that 3(Af) is formally integrable and that "^{M), 
3(M) n 3(-^) are both distributions of constant rank. Then 



(2.10) 3(M) n E3(M) C 1C3(M). 

Proof. The first claim is a consequence of the fact that the set of isotropic 
vectors of a semidefinite Hermitian symmetric form is a complex linear space. 
To complete the proof, we need to show that, if ^ G H®M, we have 
C^{Z, Z) = for all Z G 3(M) n E^{M). Having fixed ^ G H®M, we can 
argue in a small coordinate patch U about its base point p = 7r(^). By the 
assumption that ^iM) is formally integrable, we can choose real coordinates 
xi, . . . ,Xm, centered at p, such that, for a pair of nonnegative integers h,i 
with 2h + £ < m, setting Zj = Xj + ixh+j for j = 1, . . . ,h, a system of 
generators of 3(Af ) in U is given by the vector fields 



(2.11) 



'^^■ = 4+^;- fori = l,. ..,/., 



l^^+j- = 5;;-T7i7 forj = i,...,l 

Here J- = 1 (^ + i^) , l;. g X^{U) satisfies L;.(0) = 0, and 

(2.12) [Li, Lj] =0 for l<i,j <h + £. 

This is obtained by first noticing that the real vector fields in 3{M) are a 
formally integrable distribution of real vector fields. By the classical Frobe- 
nius theorem, we can choose a system of local coordinates in which this real 
distribution is locally generated by the L/i+i, . . . , Lh+i above. By linear al- 
gebra we can obtain, in a neighborhood of p, from any basis of 3{M) that 
includes Lh+i, . . . , L^^i, a new one in which the Li, . . . , Lh have the prop- 
erty that Lj — jT^ does not contain either j^ for i = 1, . . . ,h, or 



J dxj dxi 5 • • • ) ) Qx^ 



for i = 1, . . . ,£. By this choice we obtain (2.12). Let 



-I 



d , '^ , d 



(2.13) ^. = 9^ + ^"^' ^it^'^KO) = 0- 
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We can assume, by a change of coordinates, that 
94(0) 



(2.14) L,{al){0) 



dzi 



for j, r = 1, . . . , /i, i = 2h-\-l,...,m — i. 



In fact, by the formal integrability condition, it follows that 



(2.15) 



dzj 



m—£ 



i=2h+l \r=l 



dZr 






for j = l,...,h, 



are commuting vector fields, and, since they also commute with their con- 
jugates, by a change of the coordinates xi, . . . ,Xm-i we can obtain a new 
coordinate system for which (2.14) is also satisfied. Let ^ G M™ be such 
that ij{0,^) = for j = 1, . . . ,h + i, where by £j{x,£^) we indicate the 
symbols of the differential operators Lj. This means that the components 
^j of ^ are zero for 1 < i < 2h and (m — i) < i < m. By the formal 
integrability condition, there is a second degree homogeneous polynomial 
q^{x) £ C[2;i, . . . , Xm-i] such that 

(2.16) Lj{i{x,S,) + q^ix)) = 0{\x\'^) forx^O. 

Next we observe that, by identifying ^ with the corresponding element in 
TqM*", and setting v^ = {i{x,^) + q^{x)), we obtain 

= d\i{x, + qd^)){Z, Z) = ZZ{v^m - ZZ{v^m - C^{Z, Z), 

VZgX^(C/). 

Thu s, in particular, Z_Zvi^{<d) = Cj{Z,Z) > if Z G 3(M) and i € H®M. 
By (2.14), we have ZZv^{^) = ZZq^(0). Consider the expression for q^ as a 
polynomial in zi, ..., Zh,zi, ... ,Zh, X2h+i, • • • , x^n-e, 



(2.17) 



9€( 



\z,z) + 



2,0,0/ N I 1,1,0/ -N I 0,2,0/- -N , 1,0,1/ //^ 

" ' ' ^- -^ ' - ' ' (z,z) + g^' ' {z,z) + q^' ' {z,x ) 

I 0,1.1/- //■. I 0,0.2/ // //N 
+qy ■ {z,x )+q/ ■ [x ,x ), 



where x" = {x2h+i, ■ ■ ■ ,Xm-i)- The assumption that C^ > means that 
ql ' {z,z) > 0. We can add to q^ any second degree homogeneous polyno- 
mial / in C[z,x"], since Lj{f) = 0(|xp) for any such polynomial. In this 
way, we obtain a new q^, still satisfying (2.16), with the property that 

(2.18) Re{q^){x) > 0, Vx G M™. 

Fix any real valued function x ^ C^(W^) with x(^) = 1 for l^^l ^ 1) < 
x{x) < 1 in M™", and xi^) = for |x| > 2. For large r > the function 

(2.19) Ur = \/^x{x\^)e-^^'^'''^^+'^^^''^^ 
belongs to C^{U). We have 



(2.20) 



m. 



tIIo 



-2^5«W^2(^)^^< / xHx)dx, 



because of ( |2.18[ ). If Z G 3(M), we have: 

\Z{ur)\' = ^^\-Tx{x^/2)Z{v^) + ^[Z(x)](x^)|'e-2-R<=''e(-) 
<Co^/^■T^\x\'^x{x^/?/2) forr»l. 
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with a positive constant Cq. Indeed xix\^/'^) = 1 when Z{ut-) ^ 0, and 
we used the fact that Z{v^) = 0(|xp) and that Z{x) = Od^^P)- Computing 
||Z(mt-)||q by making the change of coordinates y = x-y/r shows that, with a 
constant Ci > independent of r, 

(2.21) \\Z(ur)\\l<Ci\f^ for r>l. 

On the other hand, for u G C^{U), we have, with a constant C2 > 
independent of u, 

(2.22) \\Z{u)\\l > \\Ziu)\\l - Re([Z, Z]{u)\u)o - C2\\u\\l 

For u = Ur and by using, while computing the integral, the change of vari- 
ables y = x^/T, we obtain 

-Re{[Z,Z]{ur)\ur)o = T f C^{Z , Z)e-^'''^^^y^ xHv / V^)dy 

+ 0(^) forr>l. 
Since x^{y/ \/t) is increasing with r, we get, with a constant C3 > 0, 

(2 24) -I^e([^'^]K)l"-)o > r J C^iZ,Z)e-'^'^'^i^yh\y)dy - C3\t\I 

for r > 1. 



By ( [230l ) and ( [23T| ), (^J+[ ll^j(wr)|lo + hllo) is O(V^) for r ^ 00. Thus, 
by ([232]) and ([234]) we obtain that £^{Z, Z) = for Z G 3(M)nE3(M). D 

Remark 2.12. By a slight variant of the proof of Proposition |2.11| we 
obtain: Assume that'5(M) is formally integrahle and that'^(M) and'5{M)ri 
3(M) are both distributions of constant rank. Let p £ M. If there exists 
^ G HpM such that C^ is definite on a complement of ZpMn ZpM in ZpM, 
then 3(Af ) is not subelliptic at p. 

Proposition 2.13. For every distribution of complex vector fields ^{M), 
we have 

(2.25) QsiM) C 1C3(M). 

Assume that 



(2.26) no{p) = dimuiZpM + ZpM) 

(2.27) 6o{p) = dimM {H^M) , 

(2.28) i^oip) = dimc{Z{p) G ZpM \ Z G K^iM)} 
are constant in M . Then 

(2.29) e3(M)=K3(M). 

Finally, if ^{M) is formally integrable and of constant rank, 

(2.30) E3(M) = 3(M) + 1C3(M), 

(2.31) A3(M) = {Z + Z| ZG]fC3(M)}. 
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Proof. By the definition of 9 3(M), we have C^{Z, Z) = for ah ^ e H®M 



and Z e 93(M). Thus ( |2.25[ ) is always vahd. 

To prove that, under the additional assumptions, we have the equality 
(|2.29), we apply an argument similar to the one employed in [12{ Theorem 



2.5]. 

By the constancy of fio{p), the characteristic set H^M is a smooth real 
vector bundle on M. Then the assumption that 6o{p) is constant implies 
that H®M generates a smooth linear subbundle of H^M, and therefore the 
quotient H^M/{H®M) is a smooth real linear bundle on M. 

Since i'o(p) is constant, 

M Bp^ KpM = {Z{p) £ ZpM I Z e K3(M)} 

is a complex vector bundle KM on M. The map S, -^ ^^\km is injective from 
the quotient bundle H^M/{H®M) to the bundle Herm(irM) of Hermitian 
symmetric forms on KM. We denote by LM the image bundle. 

The dual bundle Herm* (ETM) of Herm(i('M) is the real linear subbundle 
of KM®mKM generated by the elements of the form Z{p)<^Z(p), for p £ M 
and Z{p) G KpM. The annihilator bundle L^M of LM in Herm*(ifM) 
contains, for all p G M, positive definite elements of Herm*(i^M). Since 
the positive definite elements of L^M form an open set in L^M, it is easy 
to construct a global section 3 of L^M with ^{p) > for all p G M, by first 
constructing local sections, and then patching them together by a partition 
of unity. If po £ ^ and Z £ 1C3(M), with Z{pq) 7^ 0, by the standard 
Gram-Schmidt orthogonalization process, we can find a smooth function 
(j) G £{M) with (p{po) 7^ and sections ^2,...,^^ G M.^(M), such that, 
by setting Zi = cp ■ Z , we have ^(p) = X]j=i ^jip) '^ ^jip) for p in an open 
neighborhood U of po in M, where k is the rank of the complex bundle KM. 
The equality 

k 
Y, ^^Z,, Zj) =0 Vp G [/, Ve G i?°M 



implies that 






By repeating this argument for a set of elements of M.^{M) whose values at 
po give a basis for Kp^M, we prove that every Z G ]K3(M) coincides, on an 
open neighborhood of po in M, with the restriction of an element of 03(M). 
Moreover, the number of elements of 0^{M) that are needed in ( |2.3[ ) does 
not exceed u = k"^ — k. Thus, if Z G ^-^{M), we can find an open covering 

{Ua] of M and, for each a, some .^2 ' • • • ' ^i'" ^ ©sC^)) foi' which 

V 

[Z, Z] + J^[zf Uf ^] G 3(C/a) + 304), Va. 
i=i 

We can assume that the covering {UaS has a finite index. Then, by using 
a partition of unity {xa} subordinated to {C/a}, and summing together vec- 
tor fields Xa ■ Z"' with disjoint supports, we end up with a finite subset 
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Zi, . . . , Z^ of vector fields in 3iM) such that 

r 



0, 



J=l 



showing that Z G &^{M). 

When moreover 5{M) is formally integrable and has constant rank, we 
obtain (2.30) as a consequence of Lemma 2.5 and Proposition 2.11 D 



Definition 2.14. We say that a higher Levi form concavity condition is 
satisfied at the point p E M if 

V^ G H^M \ {0} with £5 > 0, 

3Zo G 3(M), Zi,...,Zr£ K3(M) + K^(M) 
withie([Zi,...,Z„Zo])/0. 



(2.32) 



Definition 2.15. We say that the distribution 3(Af) is regular at a point 



Po G Af if its rank, and the functions fio{p) of (2.26), 5o{p) of (2.27), fo(p) 



of (2.28), are all constant in an open neighborhood U of po- 



Since the rank of 3(Af), and the functions /xq and z^o ^rs all integral 
valued and semi continuous, and 60 is semicontinuous on the dense open 
subset where no is constant, the set of regular points of '3{M) is open and 
dense in M. 



Proposition 2.16. Let po be a regular point for "^{M) . Then (2.32) implies 
(1.21). If in addition we assume that '^{M) is formally integrable, then the 
two conditions (2.32) and (1.21) are equivalent. 



Proof. The regularity assumption implies that all ^ G Hp^M with ker£j D 



Q^{M) belong to the linear span (^H® M'^. Thus condition (2.6) holds if 



i G H^^M\{{)} andker/:^ D Q},{M). Whenker£^ 7^ 03(M), the restriction 
of £g to Q^{M) is semidefinite, and hence there are Zi,Z2 G Q^{M) with 

£^(Zi,z2) = ie([^i,^2])/o. 



that (2.32) implies condition (1.21) at regular points oi'^{M). 



Then because of Propositions 2.7 and 2.13, and Lemma |2.5[ we obtain 



If in addition '^{M) is formally integrable, then by the equality (2.30), 
the opposite implication is also true. D 

3. PULLBACKS OF DISTRIBUTIONS 

Let M, A^ be smooth real manifolds, and N — > M a smooth submersion. 
Given a distribution of complex vector fields 3(-/W^), its pullback [vj*^\[N) 
consists of all W G X^{N) with the property that 

(3.1) Vt/°P'"' C M,Vct G C°°(C/,iV), with woa = id^/, dw^{Woa) G ^{U). 
Let 

(3.2) V^N = {v eTN \ dw{v) = 0} be the vertical bundle, and 

(3.3) QJ"(iV) = C°°(iV, y^iV) the vertical distribution. 

Lemma 3.1. Let N — > M be a smooth submersion. Then 

(3.4) ^^{N)c[uj*3]{M) and [^^{N),[nj*3]{N)] C [uj*5]{N). 
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Proof. We note that [n7*3](-^) is the space of global sections of a fine sheaf 
of left C°°-modules. Thus by localization we can reduce the discussion to the 
case where N = M x Q, for an open subset fi of a Euclidean space M , and 
zu is the projection onto the first factor, in which situation the statement is 
trivial. D 

Definition 3.2. li N —^ M is a smooth immersion, we define the puUback 
[i*3]iN) of 3(M) to N to be the set of complex vector fields Z' G X'^(iV) 
having the following property 

J Vgo e N, 3y°P"° C N with V 3 po, and Z e 3(M) 

^^■^^ \ s.t. di{q){Z'{q)) = Z{i{q))yqeV. 

Let M, A^ be smooth manifolds. A smooth map A^ — > M is a submersion 
onto its image if there exists a smooth manifold S, a submersion A^ — > S 
and an immersion S —^ M that factorize 0, i.e. that make the following 
diagram commute: 

N -^ M 
(3.6) 

S S 

Definition 3.3. The puUback of 3{M) by a map A^ — > M, which is a 
submersion onto its image, is the distribution of complex vector fields 

(3.7) [cl>*^]{N) = [w*[i*MN), 



where w and l are the maps in (3.6) 



Remark 3.4. If N is an open subset of M and i : A'^ ^-> Af is the inclusion, 
then t*(3)(A^) = 3(A'^) is the distribution on A^ that is generated by the 
restrictions to A'" of the vector fields Z G 3{M). 

More generally, if A^ C M is a smooth submanifold, and l : N ^^ M the 
embedding map, then [i*3](A^) is the distribution generated by the restric- 
tions to A^ of the vector fields Z G 3{M) with Z{p) G TpN for every p ^ N . 

Definition 3.5. Let M, A^ be smooth manifolds, and 3m(-^^), '5n{N) dis- 
tributions of complex vector fields on M and A^, respectively. Let N —^ M 
be a submersion onto its image. We say that (/> is a Z-morphism if 

(3.8) 3n{N) C [0*3m](M). 



Remark 3.6. We keep the notation of Definition 3.5 and let zu, i be the 



maps in (3.6). Set 35(5') = [/-*3a/](5'). Then A^ — > M is a Z-morphism if 



and only if A^ — > S* is a Z-morphism. 

Lemma 3.7. Let M,N he two smooth real manifolds, and N — > M he a 
smooth suhmersion. Then 

(3.9) E^.3,,(A^) = [zz7*E3,J(A^) 

(3.10) IC„.3,,(A^) = [zz7*K3,J(Ar) 

(3.11) Q^^^,,{N) = [w*Q^J{N). 
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Proof. Again the statement becomes trivial after, by localization, we reduce 
to the case where A^ = Af x $7 with $7 open in M and w being the projection 
onto the first factor. D 

Proposition 3.8. Let M and N be smooth real manifolds, with assigned 
complex valued distributions of smooth complex vector fields 3m (Af) cind 

3Ar(iV), respectively. Let N —^ M be a smooth submersion onto the image 
and a Z-morphism. Let qo G N andpo = 4>{qo) be a regular point o/3M(Af); 



according to Definition 2.15. Assume that 



(1) 3Af(-A) satisfies the higher Levi form concavity condition (2.32) at 
the point qo; 

(2) the pullback (jf : H^ M -^ H^ N is injective. 



Then conditions (2.32) and (1.21) at po are valid for ^]\j{M). 



Proof. Using Remark 3.6 we shall split the proof by separately considering 
the case in which (/> is a submersion and the case where (p is an immersion. 

First we assume that A — > M is a smooth submersion. In this case 
^3iv(^) "^ ^^</>*3a/ (-^)- Moreover, (2) is automatically satisfied because 
3Ar(A^) C [0*3a/](-A). The statement is trivially true, as it is easily checked 
by reducing it to the case where A = M x il, with il open in M'^ and w the 
projection onto the first coordinate. 

To complete the proof of the general case, it suffices, by localization about 
qo, to consider the case in which A^ = S" is a smooth submanifold of M, 
and 3iv(A) are the restrictions to A of elements of 3m (Af) with real and 
imaginary parts tangent to A at all points of A. By (2), ( 2.32[ ) for 37v(A^) at 
Qo = POi implies that (2.32) is satisfied at po for 3m(-^)- By the regularity 
assumption, this implies (1.21) at po for ^m{M). D 



4. HyPOELLIPTICITY for SOME DIFFERENTIAL OPERATORS OF THE 
FIRST AND OF THE SECOND ORDER 



We keep the notation of ^ ^ Theorems 4.1 , Corollary 4.2 and Theorem 



4.3 below, which concern systems of first order partial differential operators, 
and second order partial differential operators closely related to sums of 
squares of vector fields, directly follow from the assumption that 3(Af ) be 
subelliptic. The other results of this section, namely Theorems |4.4| and |4.8[ 
refer to generalized parabolic second order operators, and are proved under 
conditions ( |4.26 ) and (4.43), respectively, that more directly involve the Lie 
structure of 3(Af ) with respect to some generalized time vector field. 

Let E —^ M he a complex vector bundle of rank r on M, endowed with 
a C-linear connection 

(4.1) V ■.X{M)xC°°{M,E)^C°°{M,E). 

In a local trivialization E\ij ~ f7 x C^, the connection V is described by the 
datum of a g[(r, C)-valued smooth one form 7 = {^9) G C°°{U,Ql{r,C) 
T*M). Using upper Greek letters for the components of the sections in 
C°^{U,E) ~ C°°([/,C'), we have 



(4.2) {Vx{a)r = Xa" + Y,l"p{X)a^ 

/3=1 



for a = 1, 



, r. 
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By C-linearity, we can define, for each complex valued vector field Z G X , 
a linear partial differential operator 

(4.3) Vz : C'^^iM.E) Ba^ VRez(^) + iVj^zi^) G C°°(M,S). 

Let us fix a smooth Riemannian metric g on M and a smooth Hermitian 
metric h on the fibers of -E — > M. Then we can define the formal adjoint 

V*z : C^iM, E) -^ C°^{M, E) by 



, h{V*zU,v)dXg = I h{u,Vzv)dXg 
\/u,v G C°^{M,E), with supp(u) n supp(v) (s M, 

where dXg is the Lebesgue density on M with respect to the Riemannian 
metric g. 

We obtain an analogue of |12| Theorem 4.1]: 

Theorem 4.1. Let E —^ M he a smooth complex vector bundle of rank r 
on M and V a C-linear connection on E —^ M. If ^{M) is subelliptic at a 
point pq G M, then any weak solution u G Lf^^{M,E) of 

(4.5) Vzu G C'^iM, E) VZ G 3(M) 

is equal, a.e. in an open neighborhood U of pQ in M, to a smooth section 
ofE^M. 

Formula ( |4.5| ) means that for every Z G 3(Af ) there is a smooth section 
fz eC°°{M,E) such that 



h{u,V*zv)dXg = h{fz,v)dXg, 

(4.6) Jm Jm 

yv G C'=^{M,E), with supp{v) (s M. 

Proof. The statement is local. Therefore by substituting for M a relatively 
compact open neighborhood oipo, we can assume that 3(-M') is generated by 
a finite set of vector fields Li, . . . , L„, and that, for some e > and C > 0, 
we have the estimate 

(4.7) \\vt<C\^^\\L,{v)\\l + \\v\\lY yv€C^{M). 



We can also assume that E is the trivial bundle M x C" on M, so that (4.5 ) 
is equivalent to the system 

r 

(4.8) LX + 5]aJ/3(^)^^ = /"eC°°(Af), for j = 1, . . . ,n, a = 1, . . . ,r, 

with aj^ G C°°{M). Defining 

( d : C°°{M, C) -^ [C^iM, C^)]" by 



(4.9) 



0((^")a=i,...,r-) = ij-n" + ^a°^(p)n^ 



P=l I l<a<r 
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we obtain from (4.7) that, with some new constant C > and the same 
e>0: 

(4.10) Ml < c {Mv)\\l + Ml) , yvec^{M,n. 

We have, for x^C^{M), 

(4.11) (c)(xn))J = x(f(«))-+(i,(x)«"). 



In particular, if u G L^q^(M, C) is a weak solution of (4.8), we have that 
c)(xu) E [L^(M, C)] . By applying Friedrichs' theorem on the identity of 
the weak and strong extensions of a first order partial differential operator, 
we can find a sequence {v^} C C^{M, C^) such that 

(4.12) v^ ^xu in L^{M, C) and d{v„) ^d{xu) in [l2(M,C^)]". 



The subelliptic estimate (4.10) yields a uniform bound for the Sobolev e- 
norm ||f;/||e. This implies that xu£ W''{M,C^), where VF'^(M, C") denotes 
the Sobolev space of L^-vector valued functions that have L^-derivatives of 
the positive real order e. Hence u S Wi^^{M, C'). 

To show that u G C°°{M, C), we use the Sobolev embedding theorem: it 
suffices to show that u £ Wy^^{M, C) for all s > 0. Assume that we already 
know that this is true for some sq > 0. If X ^ Co°(^)) then 

Hxu) = [d,x]{n) + xHu) G W{M,C1 

and has compact support. Fix any scalar pseudodifferential operator Ag G 
^''(M), with sq — e < q < sq. Then, by the continuity properties of classical 
pseudodifferential operators, we obtain 

(4.13) HMxu)) = [d,Ag]{xu)+Ag{d{xu)) G ^^^.(M,^), 

because Ag and the commutator [d, Ag](n) have orders < sq and x u, d{x u) G 
iy^"(M) and have compact support in M. Thus, by the argument above, 
Agixu) G W'{MX'')- This implies that X M G Ty5+'(M,C''), with g + e > s. 
Since x was an arbitrary smooth function with compact support in M, this 
yields u G ^[^^'(M,^). By recurrence we obtain that u G Wf^^iM,^) 
for all s > 0, and hence is equal a.e. to a smooth section. The proof is 
complete. D 

Corollary 4.2. Assume that M is compact and that '5{M) is subelliptic at 
all points p £ M . Then 

(1) the space 

(4.14) 0^{M) = {ae C^iM, E) \ Vz(a) = 0, VZ G 3(M)} 

is finite dimensional. 

(2) The map 

(4.15) C'^iM, E) X 3(M) 9 (a, Z) ^ {Vz{<y),Z) G C^i^M, E) x 3(M) 

has a closed range. By this we mean that, if {ctu] is a sequence of sec- 
tions in C°°{M,E) and for each Z G 3(M) there is fz G C°°{M,E) 
such that VzicTu) converges uniformly to fz in M, then there exists 
a section a G C°°{M, E) with Vz(o-) = fz for all Z G 3(M). 
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Proof. (1) follows from the Sobolev embedding theorem, because 0-^{M) 
is an L^-closed subspace of C°°{M,E) on which the L? and the e-Sobolev 
norm, for some e > 0, are equivalent. Finally, (2) is a consequence of the 
fact that, since M is compact, for a finite set Li, . . . , L„ G 3(-W), some e > 
and some const > 0, on the L^-orthogonal complement of 0-^{M), we have 
the coercive estimate 

(4.16) ||u||2 <const I ^||Lj(u)||g I , Vn e C°°(M,E) n [C'3(M)]^ . D 

Theorem 4.3. Let 'b{M) he a distribution of complex vector fields. 

(1) We can find a locally finite family {ij} C '^{M), such that, for any 
choice of a €z C°°{M), the second order operator 

(4.17) P{u) = au + ^LjLj{u) 

j 
is hypoelliptic at all points of M at which 3(Af ) is suhelliptic. 

(2) If'^{M) is finitely generated then, for any set of generators Li, . . . ,Ln 
ofi{M), and for any choice of Zq G 3(M)+3(M) and a ^ C°°{M), 
the operator 

n 

(4.18) P{u)=^L,jLj{u) + Zo{u) + au 

is hypoelliptic at all points p of M where '5[M) is suhelliptic. 

Let M' he the open suhset of M of points p where 3{M) is suhelliptic. 
Then the operators P of (1) and (2) satisfy the following: 

{ V[/°P"" (S M', 3e > 0, C7 > such that 
(4 19) < 

\\\u\\l<C{\{P{u)\u)o\ + \\u\\l), WuGC^iU). 

Proof. Let {Ui,} be an open covering of M' by relatively compact open 
subsets, and, for each of them, let Li' , . . . , LnJ G 3(-/Vf ) be chosen in such 
a way that, for suitable e^ > 0, Cu > 0, we have the estimate 

(4.20) h||2_^ < a (fl ll4''Wllo + ll^llo j , Vn G C^{U,). 

Take smooth functions Xu G C^{Uu) such that supp(X[y) <e U^, the family 
{supp{xu)} is locally finite, and J2u \Xu{p)\'^ > for all p G M'. Then (1) 
holds with {Lj} = {xuLI^^}- 



By [19], the hypoellipticity of (4.17) and of (4.18) is a consequence of 



(4.20). Since it suffices to prove that for each p G M' , there is a small 



open neighborhood U (s M' of p for which (4.20) holds true, we can re- 
duce the proof to the case where 3(Af) is the C°°(M)-module generated by 
Li, . . . ,Ln G 3{M), and the operator P is of the form ( |4.18 ). 
By integration by parts we obtain 

n 

-{Pu\u)o = ^ ||Lj(u)||^ - (Lo(n)|u)o + (n|L„+i(n))o + ia'u\u)o 



(4-21) 



WueC^iM). 
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Since Li, . . . , L„ generate 5{M), we obtain that (0.1 ) is valid for every rela- 
tively compact open subset U (s M' , and this in turn, together with (4.21), 
implies (4.20). The proof is complete. D 



Theorem 4.4. WekeepthenotationofDefinitionl.il Assume that ^{M) 
is generated by a finite set Li, . . . , L„ of complex vector fields. Let Xq G 
X(M) be a real vector field. Let 

(4.22) To(M) = C'^{M,R) ■ Xq + T^{M) 

(4.23) T;(M) = [2t3(M),r,_i(M)], forh>l, 



(4.24) 



t'(m) = £tUm). 



h=0 



Ln particular, T'(M) is the %^[M)-Lie-submodule of X{M) generated by Xq 
and T3(M). Then, for any choice of Yq G T^°\m) and a G C°°(M), the 
second order partial differential operator 

n 

(4.25) P{u) = a-u + Xo{u) + iYQ{u) + ^ LjLj{u) 

is hypoelliptic at all points p £ M where 

(4.26) {X{p) I X e T'{M)} = TpM. 

We divide the proof of Theorem |4.4| into several steps. First we prove 



(4.27) 



u 



Lemma 4.5. Let U (s M be an open set and assume that there are e > 0, 
C > and Ai,. . . ,Ar € ^°(C/) such that 

( n / r \ 

'2 + J] \\L,{u)\\l < C ^ |(P(n)| A(n))o| + ||n||g , 
j=i \h=i J 

V-u G C^{U). 

Then for every real s > 0, and every open subset U' with U' d U , there is a 
constant C = C{s, U') such that 

n 

(4.28) \\u\\l^, + ^ \\L,{u)\\l < C {\\P{u)\\l + ||n||g) , Vn G C-([/'). 
i=i 

Proof. Let A^ G ^^([7) be elliptic. Then we have, with real constants 
Ci > 0, C2 > 0, uniformly for u G C^{U'), 



n 



n 



U 



,.+, + Yl WH^n's < Ci \\As{u)f, + ^ ||A.(L,( 

n 



U))|lo 



<C7i ||A,(«)||2 + ^||L,-(A,( 

Thus, using the inequality 

r V5 > 0, 3C5 > s.t 

1 \Ml<sMU. + Cs\\u\\l yuec^iu) 



ujjwl I +C2\\uf 
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we obtain that the left hand side of (4.28) is bounded by a constant times 

r 



Y,\iPiUn))\Ah{As{u)))o\ + \\u\\l 



h=l 

The operator Q = P + X]?=i ^*j^j ^^ ^^ operator of the first order, with 
principal part Xq + IYq. Therefore if A E "^^{U), we obtain 

{P{Uu))\A{Uu))o = {UQiu))\AiUu)))o + 0{\\u\\l) 

n 

-^(L,(A,(n))|L,(A,(A(t.))))o 
= {As{Q{u))\A{As{u)))o + 0{\\u\\l) 

n 

+ J2iLjAsLj{u)\A{As{u)))o + 0{\\Lj{u)\\s\\u\\s) 

n 

+ Y.{[As,L,]{u)\[L„AoA,]{u))o 

n 

+ Y,i[^s,Lj]{u)\AoA,{L,{u)))o 



i=i 






= {A,{P{u))\A o A,(n))o + 0(||n||f + J^ \\L,{u)\\ 

where we use 0{N(u)) to indicate some quantity whose modulus is bounded 
by a constant times N{u). This computation yields 



(4.29) < 



J=l 



\h=l 



'OO fTT'\ 



yueC^iU'). 



Clearly (|429|) implies ( |4.28[ ). 

It is known (see e.g. [HI |T8]) that 



D 



Lemma 4.6. // (4.28) is valid for all s G M+ and all open subset U' d U, 
then P is C°° -hypoelliptic in U. 



End of the proof of Theorem \4-4 By the previous Lemmas, we only need to 

it, for all u 

Xo{uu)dXg 



prove ( |4.27D . First we note that, for all ueC^{U), 

1 
2 



|Re(Xo(n)|n)o| 



u 



u 



u\'X*o{l)dX, 



1 



<- sup|Xo*(l)| \Mt, 



.pdU 
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\{iYo{u) + au\u)o\ < Co\\u\\o \\u\\o + J^ \\Lj{u)\\o , Vu G C^{U), 
we obtain, upon integrating by parts, that with a constant Ci > 0, 

n 

(4.30) ^||L,(n)||2<-2(P(n)|n)o + Ci||tx||2, yuGC^{U). 

i=i 

Next we note that, taking for Aq £ "^^(U) the composition with Xq of 
an elhptic pseudodifferential operator A_i G \I'~^(C/), we have, with some 
constant C2 > 0, 

\\Xo{u)\\\ < C2 (|(Xo(n)|^oN)o| + Ml) , \/u G C^{U). 



We obtain, with constants C3, C4 > 0, 



\{Xq{u)\Ao{u))\<\{P{u)\A^{u))o\ + 



^(L,(n)|L,(^o(n)))o 



j=i 



u\\l^ 



+ C^\M ||u|| + ^||Lj(n)||o 

n 

<\{P{u)\Ao{uM + Y,\\L,{u)\\l + C^\ 

V-u G C^(C/). 

To complete the proof, it suffices to show that, if y G X(M) satisfies, for 
some (5 > 0, for some Ai,...^Ar G ^''(C/), and a constant C" > 0, the 
estimate 

(4.31) ||y(tx)||i_i < C \J2 \{P{u)\Ah{uM + ||n||2 j , Vn G Co°°(C/), 

and X G A3(M), then we have, with some A'^, . . . ,A'^, G '^^(U)^ and con- 
stants 5' > 0, C" > 0, 



||[X,y](n)|||_i <C"[Y^ \{P{u)\A'^{u))o\ + 



\u\\l , 



\h=l 



(4.32) 

VnGC^(?7). 

Recah that the estimate ( 1.13| ) holds for Z = X, and hence 

(4.33) \\X{u)\\l<cx{\{P{u)\uU + \\u\\l) yuGC^iU), 

with a constant cx > 0. Then, with positive constants Cj > and a 
Tg "^^-^{U), we get 



||[X,y](n)||L^ < CO (|([X,y](n)|r(n))o| + ||n||2) , Vn G C^{U). 
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Assuming, as we can, that (^ < gj we obtain 

\iXY{u)\T{u))o\<\{T*{Y{u))\Xiu)U + \{[T,XnYiu))\u)o\+ci\\u\\l 

< C2||y(n)||5-i {\\X{u)\\o + ||n||o) + ci\\u\\l Vn G C°°([/). 



The last term, in view of (4.33), can be estimated by the right hand side of 
( |02| ). Likewise 

\{YX{u)\Tiu))o\ < \{X{u)\TiYiu)))o\ + \{Xiu)\[T,Y]{u))o\+cs\\u\\l 

< C4||X(u)||o {\\Y{u)\\s-i + \\u\\o) + C3\\u\\l 



and in view of (4.33), also this last term can be estimated by the right hand 
side of (4.32). The proof is complete. D 



Remark 4.7. Theorem 4.4 is weaker than the analogous statement in |14] 
in the case where the Lj's are real. Indeed, for Lj G X(M), setting Xq = 
Lq, our assumption requires that all commutators [Ljj, . . . , Lj^_^, Lj^] with 
r £ Z_|_, and < j/j < n, and jh > for h < r, span the tangent space TpM. 
The statement in [Uj, also proved in [16] and [8], allows jh = also for 
1 < h < r. This motivates us to consider separately the special case where 
E,^(M) = 3(Af) + 3iM): Theorem 4.8 below generalizes the case where all 
the Lo's are real. 



Theorem 4.8. We keep the notation of Definition 1.11\ Assume that 



(4.34) 

(4.35) 

(4.36) 

(4.37) 

(4.38) 

Let us define: 

(4.39) 

(4.40) 



Li, . . . , L„ G X'^(M) generate 3(M), 



E3(M)d3(M), 
Lq is a real vector field, 



^„+ig3(m) + 3(m), 

aGC°°(M). 
M{M) = A(o)(M) = A3(M) + C°°[M, M) Lq 



A'(,)(M) = [A'(M), A'(,_i)(M)] forh>l 

oo 

(4.41) T"(M) = J;A'(,) 
Then the second order differential operator 

n 

(4.42) P{u) = Lo{u) + Y,LjLj{u) + Ln+i{u) + 

is hypoelliptic at all points p G M where 

(4.43) {X{p) I X G T"{M)} = TpM. 
Proof. We shall prove that, for every X G T"(M), 

r Vt/°P"'" (s M, 3e > 0, 3C > 0, s.t. 



au 



(4.44) 



\X{u)\\l, < C {\\iPiu)\\l + \\u\\l) , yu G C^{U). 
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We observe that the proof of Theorem |4.4| shows that, if y € X{M) satisfies 
( |4.44[ ), and X £ A^{M), then [X,Y] also satisfies ( |4.44[) (with e/2 substitut- 
ing e). Thus to show that all X G T"(M) satisfy (4.44), it suffices to prove 
the following 



Lemma 4.9. IfY e X(M) satisfies (4.44), then [Lo,Y] also satisfies (4.44). 



Proof. We closely follow the argument in |16| p. 66-68]. Condition (4.35) 
means that, for every U <£ M there is a constant Cq > such that 



(4.45) 5^||Z,(n)||g<Co 5]||L,(n)||2 + 




WueC^iU). 



Thus, by (4.30), we obtain, with a constant Ci that only depends on [/ d M, 
(4.46) 



Y^ {\\Lj{u)\\l + \\L,{u)\\l) < Ci (|(P(n)|n)o| + ||^||g) , 

Vu G C(^(C/). 



Let Y G Je(M) satisfy ( |04| ). We have, with T2S-1 G "^"^^-^U), and for all 
« G Co°°(C/), 

||[i^o,l^](n)||Li < C2 {\{[Lo,Y]{u)\T2S-i{u))o\ + \\u\\l) 

< C2 {\{LoY{u)\T25-i{u))o\ + \{YLo{u)\T25-i{u))o\ + ||u||g) . 

We shall estimate separately each summand inside the parentheses in the 
last term. In the following U will be a relatively compact open subset of M 
and all functions u will be smooth and have compact support in some fixed 
relatively compact open subset of U. 
We have 

n 

P*{u) = -Lq{u) + ^ L*Lj{u) + L',^^i{u) + au, 



with L'^^^ G 3(M) + 3(Af) and a' G C^iM). Hence using (|06|, 
\{LoY{u)\T2S-i{u))o\ < \{P*Y{u)\T2S-i{u))o\ 

^*> +Y,\mL,Y{u)\T2S-i{u))o\ + C,{\\u\\l + \\Y{u)\\ls^, + \\P{u)\\l). 

2 



The term in parenthesis can be estimated by a constant times {\\u 

|2 



+ 





P(u)IIq), provided we choose a (^ so small that Y satisfies (4.44) in U with 



some e > 2(5. For the first summand on the right hand side of M, we have 

{P*Yin)\T2S-iiu))o = {T^s-iiYiu))\P{u))o + {Y{u)\[P,T2s^,]{u))o. 

We obtain 

|(T2V,(y(n))|P(n))o| < C4 {\\Y{u)\\ls., + \\P{u)\\l) . 

The right hand side is bounded by a constant times (||u||q -|- ||-P(ii)||o)) pro- 
vided again that (4.44) holds for Y with e > 26. 
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For the commutator [P,T2S-i], we have 

n 

[P, T2S-1] = Y. T^s-iL*{u) + K-iH^) + T^Li, 
i=i 

withT^,_„r^',„,,T-_,GvI/2^-i(f/). 

Thus, because of (4.46), also the term |(y(n)|[P, T25_i](u))o| is bounded 
by a constant times {\\u\\'q + ||P(m)||q), provided again that (4.44) holds for 
Y with e > 2(5. 

We have 

with -B25-I' ^25-1' ^25-1 ^ ^^'^"H^)- Thus with a constant C4 > 0, 
|(L*L,y(n)|T25_i(n))o| < \{LjT*s-M\LMM + |(i3^5-i^HI^,* W)o| 

+ |(S^'5_iy(n)|L,(n))o| + \{B'i:,_,Y{u)\u)^\ 

< C4 (IIL.r^Vinn)!!^ + \\Hu)\\l + \\L,{u)\\l 

+ \\Y{u)\\ls-i + Ml). 

Therefore, provided again that ( 4.44[ ) holds for Y with some e > 2(5, all 
terms but those of the form ||Ljr2^_^(u)||Q are bounded by a constant times 
(ll^llo + ll-f(^)llo)- Thus we only need to bound the terms \\L jT^^_^{u)\\'q. 
We have, by (4.30), with some constant C5 > 0, 



^ \\L,n,_^Y{u)\\l < C, {\{P{T^s-iy{^))\ns-iY{n))o\ + \\ns-iY{n)\\l) . 

i=i 

The last summand inside the parentheses on the right hand side is bounded 

by a constant times (||n||o+ ||P(u)||o), provided again that (4.44) holds for Y 

with some e > 26. Let us consider the first one. Note that the composition 



'-25 



'■2(5-1 



y is a pseudodifferential operator in ^ ([/). We have the 



commutation formula 



3 = 1 



III 

25 



withF;^^,F;^^,F-GM/25(C/). 
Thus 
{P{T;,_,Y{u))\T*^,_^Y{u))^ = iPiT2siu))\T*s_,Yiu))o 

n 

= {p{u)\t;,t;,_,y{u)), + Y.^Hu)\[f'^j*t;,_,y{u))o 

+ j^{L,{u)\[FlXn5-iY{u)), + {u\[F^'srns.^Y{u))o. 
Hence we obtain, with some constant Cg > 0, 

n 

\\L^n,_,Y{u)\\l < C,{\\P{u)\\o\\Y{u)\Us-i + Y. i\\H^)\\l + ll^iMllo) 

+ \\Y{u)\\ls_, + \\u\\l), 
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which can be bounded by the right hand side of (4.44 1, provided Y satisfies 



(4.44) with e>4S. 



Finally we note that 

|(yLo(n)|r2,_i(n))o| < Cr{\{T^s-,{u)\LoY{u))o\ + \\Lo{u)gs^, 
(4.47) 

+ \\Y{u)\\ls-i + \\u\\l). 

Thus, by repeating the discussion above with T'2^_^ replacing T2S-1, we find 
that the left hand side of (4.47), provided Y satisfies (4.44) with e > 45, is 
bounded by a constant times 
the Lemma. 



u 



i + ||P(u)||q). This concludes the proof of 

D 



End of the Proof of Theorem \4.8[ By the discussion at the beginning of the 
proof, and by Lemma |4.9[ we obtain that for every relatively compact open 
subset U of M, which is contained in the open subset M' of M, consisting 



of the points p where (4.43) is satisfied, there are positive constants e > 
and Co > such that 



(4.48) 



u 



+ E (ll^:'-(^)llo + ll^-Wllo) < CO iWufo + \\P{u)\\l) , 

Vn G C^{U). 

One easily shows by recurrence that, if [/ is a relatively compact open subset 
of M, then there is a positive constant e > 0, and, for every real s > 
another constant Cg > 0, such that 



(4.49) 



k||?+. + Y. {\\H^)\\' + WH^n's) < CO {Ml + ||P(n)||2) , 

The hypoellipticity of P in U, with a gain of e derivatives, follows in a 
standard way from (4.49) (see e.g. [14J. D 



5. Applications to almost CR manifolds 

In this section we shall consider the case where M is an almost CR mani- 
fold of CR dimension n and CR codimension k, and 3{M) is the distribution 
of vector fields of type (0, 1) on M. This means that conditions (i), (ii) and 
{iii) below are satisfied: 

(■j) M has real dimension 2n + k, 

(ii) 3(-/W^) has constant rank n, 

(m) 3(M)n3W = {0}, 

(iv) [3(M),3(M)]C3(M). 

When the formal integrability condition {iv) is also satisfied, we say that M 
is a CR manifold. 

When M is an almost CR manifold, it is customary to write T^'^M for the 
complex bundle with fibers Tp' M = ZpM. The Z-morphisms of §3^ are then 
the CR maps, i.e. the smooth maps (j) : N ^ M with 
for all q £ N. 



P{t!I'^N) C T^lt.M 



nO,l 
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Lemma 5.1. // an almost CR manifold M satisfies the higher order Levi 
form concavity condition at a point p, then M is of finite type at p. D 



The next statements clarify in what sense condition (2.32) is a pseudo- 
concavity condition. 

Proposition 5.2. Let M be a CR manifold of hypersurface type, i.e. of 
CR codimension k = 1. If ^{M) satisfies the higher Levi form concavity 



condition (2.32) at a point p, which is regular for ^{M) , then M is strictly 



pseudoconcave at p. 

Proof. Let 7^ ^ G H^M. We want to prove that C^ is indefinite. Assume 
by contradiction that this is not the case. Replacing, if needed, ^ by {—C), 



we can assume that C^ > 0. By assumption (2.32) we can choose Zq G 3{M) 



and Zi,...,Zre K^{M) U K^{M) satisfying 

(*) a[Zi,--.,Zr,Zo])^0. 

We can take r minimal with this property. In particular, we have 

(**) [Z2,...,Zr, Zo]{p) = Z{p) + W{p), 

with Z,W e 3(M). Assume that Zi G ^(M). Then ^{[Zi,Z]) = 0, 
because of the integrability condition {iv), and then, from M, we have 
C^{Zi,W) / and C^{Zi,Zi) = 0, yielding a contradiction. Likewise, 
if Zi G IK3(M), we have ^([Zi,W^]) = by {iv), and hence, from ([*]), 
C^{Z,Zi) / 0, while £g(Zi,Zi) = 0, contradicting the assumption that 
C^ >0. The proof is complete. D 

Corollary 5.3. Let N be a generic CR submanifold of a CR manifold M 
(this means that Tp' N = Tp' M D T9N and the restriction map H^M — > 
H^N is infective for all p £ N C M). If M is of hypersurface type and 



3(iV) satisfies the higher Levi form concavity condition (2.32) at a point po, 
regular for ^{M) , then M is strictly pseudoconcave atpQ. 



Proof. The statement follows from Propositions 3.8 and 5.2 D 



Lemma 5.4. Let M, N be almost CR manifolds, and w. N ^ M a CR map 
and a smooth submersion. We denote by ^{M) and 3(-^) the distributions 
of (0, 1) vector fields on M , N , respectively. If M is strictly pseudoconvex 
at a point pQ, i.e. if there is ^0 S Il2„M with £f„ > 0, and moreover po is 



po 



regular for "^{M), then the higher Levi form concavity condition (2.32) for 
3(A^) is not satisfied at any point qq G zu~^{pq). 

Proof. Replacing M by an open neighborhood of po in M, we can assume 
that M is strictly pseudoconvex at all points. Then 'K^(M) = 0, and 
hence 1^3 (A^) is contained in the complexification of the vertical distribu- 
tion QJ^(A). Indeed, if ^ G i?®M and g G zu-^ip), the pullback 'uj*{q){^) 
belongs to H®N. 



Thus if r/o = '^*{qo){S,o) for 90^^ ^(Po), then 770 does not satisfy (2.32). 



Indeed, rjo vanishes on the pullback of 3(Af) -|- 3(Af), and this distribution 
is a 53'^(A)-module. D 

Theorem |4.1| and Corollary |4.2| yield 
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Theorem 5.5. Let M be an almost CR manifold and assume that condition 



(1.21) is satisfied by the distribution 3(-M) of its (0,1) vector fields at all 
points of M . Then 

(1) If E —^ M a Hermitian vector bundle on M , endowed with a C-linear 
connection V, then all weak solutions u G Lf^^{M,E) of 

Vz{u)eC'^{M,E), "^ZG^iM) 
are smooth sections of E —^ M . 

loc 



In particular, all CR sections of E {i.e. weak Lf^^ solutions of 



Vz{u) = 0, for all Z G 3(M)) are smooth. 

(2) In case M is compact, the space of CR sections of E is a finite 
dimensional C-linear space. 

(3) If M is a compact CR manifold, then the cohomology groups H^' (M), 
for p = 1, . . . ,n + k, of the tangential Cauchy-Riemann complexes, 
are Hausdorff. D 

6. SUBELLIPTICITY CONDITIONS FOR HOMOGENEOUS CR MANIFOLDS 

Let M be a CR manifold, homogeneous for the CR action of a Lie 
group G. Fix a base point o £ M and denote by 

(6.1) w:G3g^g-oeM 

the associated principal bundle. In [20j we associated to M and the base 
point o the CR algebra (g, q), that is the pair consisting of 

- the Lie algebra g of G, 

- the complex Lie subalgebra q of g = g + ig given by 

(6.2) q = [dzUe]-HT^''M), 

where [dvuelf^ '■ g^ — > T^M is the complexification of the differential 
dzue : g — > ToM of w at the identity. 

For X G g we denote by X* the left-invariant vector field in G with X*(e) = 
X. Let 

(6.3) q* = {X* + iY*\X,Y £Q, X + iY£q}, 

(6.4) 3(G) = £{M) q* = the vector distribution spanned by q*. 

The following statement is straightforward. 

Proposition 6.1. Let M = G/Go be a G-homogeneous CR manifold, 
3(M) the distribution of (0,1) -vector fields on M , and^{G) the distribution 



on G defined by (6.4). Then 

(1) The distributions 3(M) and 3(G) are regular at all points; 

(2) the principal bundle fibration G — > M is a Z-morphism, 

(6.5) 3(G) = [tz7*3](G) 



and conditions (2), (3), (4) of Proposition 3.8 are satisfied. D 



Proposition |6.1| can be used to reduce the question of the subellipticity of 
the distribution of (0, l)-vector fields of a homogeneous CR manifold to Lie 
algebra computations. 
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Let (g, q) be the CR algebra associated with the G-homogeneous CR 
manifold M and its base point o. Let 

(6.6) 1° = H^^G = {e e 5* I e(Re(Z)) = 0, VZ G q}. 
To each ^ G t'' we associate the Levi form 

(6.7) C^{Z, W) = i^{[Z, W]), for Z,W € q. 
We also set 

(6.8) i® = {e e i° I A > 0}, 



(6.9) 



«q = {ZGq|£.(Z,Z)=0, Veet®}. 



Since all points of M and of G are regular for 3(-^) and 3(G), respectively, 



we obtain from Proposition 2.13 



Lemma 6.2. Let (g, q) be a CR algebra. The set tq is a linear subspace of 
q and is equal to the set 



Zoe q 



3Zi,...,Zr£q, s.t. ^[Zj, Zj] G q + q 
j=0 



The elements 

Z* = {Re Zy + i{lm Zy , forZeiq, 
generate the distribution 1C3(G), that is equal to 03(G). 



n 



As a consequence of Propositions 2.16 6.1 and Lemma |6.2| we have 



Proposition 6.3. Let M = G/Gq be a homogeneous CR manifold and let 
(g, q) be the CR algebra associated with M and the base point o. We denote 
by 3(Af) the distribution of (0,1) -vector fields on M. Then the following 
are equivalent. 



(1) ^{M) satisfies condition (2.32). 



(2) ^{M) satisfies condition (1.21). 



(3) 3(G) satisfies condition (2.32 



(4) 3(G) satisfies condition (1.21). 
(5 

(6.10) 



(5) (g, q) satisfies the condition: 

J V^ e t® \ {0}, 3Zo e q an(i Zi, . . . , Z^ G «q U !q 
I s.t. ie([Zi,...,Z„Zo])/0. 



D 



7. Orbits of a real form in a complex flag manifold 

In this section we investigate the subellipticity of the distribution of the 
(0, l)-vector fields of the homogeneous CR manifolds which are real orbits of 
real forms in complex flag manifolds. The study of their CR geometry has 
been already started in [U [2] , to which we refer for the complete explanation 
of many details. 



COMPLEX VECTOR FIELDS AND HYPOELLIPTIC PDO'S 31 

7.1. Complex flag manifolds and orbits of a real form. We recall that 
a complex flag manifold is a closed complex projective variety, that is a coset 
space of a connected semisimple complex Lie group G with respect to a 
complex parabolic subgroup. 

A real form, G of G is a real Lie subgroup of G whose Lie algebra q 
is a real form of g"--, i.e. such that C g^ and g^ = g + ig. We shall write 
Z for the conjugate of an element Z G g with respect to the real form g. 
The left action of G decomposes F into a finite set of G-orbits (see p^). 
All G-orbits M = G • o are generically embedded CR submanifolds of F. 
It turns out that F and also M, if we take G connected, are completely 
determined by the Lie algebra g , by its real form g, and by the complex 
parabolic Lie subalgebra q C g^ of the isotropy subgroup Q at o. Thus we 
shall write M = M(q, q), for the homogeneous CR manifold M. We denote 

by Go = Q n G the isotropy subgroup at the base point o and by G > M 

Go 

the principal Go-bundle w. G B g ^ g ■ o €z M. 

The Lie algebra go of Go contains a Cartan subalgebra f) of g (see e.g. 
JMIIIIIS])- Its complexification i)'^ = [} + ii) is a Cartan subalgebra of g^. Let 
TZ be the set of roots of g^ with respect to f)*^, and g^ = {Z G g*^ | [H, Z] = 
a{H)Z, \/H e f)^} the eigenspace corresponding to the root a (^ TZ. Since 
f) C q, the subalgebra q is adgc(f)^)-invariant. Hence we have 

(7.1) q = [^Cg^^gC^ fo^ 

(7.2) Q = {q G 7^ I g^ C q}. 

To say that q is parabolic means that Q contains a positive system of roots 
7^"*". Let ^ be a corresponding partial order on the linear span fij^ of TZ, 
and B = {ai, . . . , a^} the set of simple positive roots in TZ^. Every root 
a £ TZ can be written in a unique way as a linear combination with integral 
coefficients of the elements of B: 

e 

(7.3) a = ^kjaj, 

j=i 

where all /cj's are either > 0, or < 0, according to whether a is positive or 
negative with respect to -<. We define the support of a to be the set 

(7.4) supp(a) = {aj eB\kj j^O}. 
Let 

(7.5) $ = {a e e I -a ^ Q} 

be the set of simple roots a whose opposite (—a) does not belong to Q. 
Then 

(7.6) Q = 7^+U{a -< I supp(a)n$ = 0}. 

Since Q is completely determined by <I>, we shall write Q$, q^, Q(j) for the 
parabolic set of roots, the complex parabolic subalgebra and the complex 
parabolic subgroup, respectively, that are attached to any special choice of 
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the subset $ of B. We shall also introduce the notation 

Ql = {ayO\ supp(a) n $ / 0} 

(7 7) 

^ ^ Q"^ = {a \ a, -a £ Q} = {a £ n \ supp(a) n $ = 0}. 

The conjugation in g^ induced by the real form g defines, by duality, a 
conjugation a — > a in 7^. We partition 7^ into three subsets: 



(7.8) 


7^rc = {a G 7^ 


a = a} 


real roots, 


(7.9) 


7^im = {aen 


a = —a} 


imaginary roots, 


(7.10) 


7^c^ = {a G 7^ 


a^±a} 


complex roots. 



The Cartan subalgebra f) is invariant under a Cartan involution ■& of g, 
whose set of fixed points Ms a maximal compact Lie subalgebra of g. With 
p = {X I "diX) = —X}, we have that for imaginary q, the eigenspace g^ is 
contained either in t'^ = t -\- it, or in p^ = p + ip. In the first case we say 
that Q is a compact root. Compact roots form a root subsystem TZ, of TZ. 

The G-homogeneous CR structure of M is defined, as in ^ by assigning 
the subspace 

(7.11) ry = dtz7(e)(q). 

Here e is the identity of G and we denote by the same symbol dzu^e) the 
complexification of the differential dw{e) : g = TgG — > TqM. 

For each Z G g we can consider the fundamental vector field Z* G 
X^{M). Its real and imaginary parts are the infinitesimal generators of the 
flows associated to the left translations by exp(tRe(Z)) and exp(tIm(Z)), 
respectively. 

Fixa Chevalley basi^ji/o, | a G B}U{Za \ a G TZ} of g*^. The restrictions 
to G of the vector fields Ha, for a £ B, and Za, for a G 2$, form a basis 
for the puUback 3(G) of 3(M). 

Since g*^ is semisimple, the Killing form k c is nondegenerate. Thus, as 
in [H §13], we can use the Killing form to identify the complexification t'^ 



of the space t" of (6.6) with the linear span of 

(7.12) Za for a G Q| n Q|. 
This is obtained by associating to Za the linear form 

(7.13) fa-Q^ ^ Z ^ Kgc(Z_a, Z) = trace (adgc(Z_a) o adgc(Z)) G C. 
Correspondingly we obtain the complexified Levi forms 

(7.14) La{Z, W) = ifa{[Z, W]), for Z,W £ q$. 



When a G T^re H Q, it actually corresponds to a Levi form (2.7) at o. 



The intersection Q^ n Q^ does not contain imaginary roots. To a pair of 



This means that Za G gJi for all a G 7?. and that, for a £ B, we also have [-ffa, Z^] = 
2Za, [Ha,Z-a] = —2Z-a, [Za,Z-a] — —Ha', and moreover that the linear map defined 
by Ha —>■ —Ha and Za -^ Z^a is an involutive automorphism of the Lie algebra g . 
Moreover, Za £ g when a G 7?.re and, if a G 7?.im, we have Za = Z^a when a is compact 
and Za = —Z-a when a is not compact. In general, Za = taZa, with ta = ±1, for all 
roots a £TZ, and we can take ta = 1 when a G TZra (see e.g. [5]). 
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complex roots a,a G Q^ n QJ, correspond the two Hermitian symmetric 
forms obtained by polarization from the Hermitian quadratic forms 



[ReL„] iZ,Z) = - (fa{[Z,Z]) - U[Z,Z]) 
1 



[ImL„] {Z,Z) = - (fa{[Z,Z]) + U[Z,Z]) 



Note that La{Z, W) = ±La{W, Z), when Za equals ±Za, respectively. Thus 
each Levi form can be written as a linear combination of the L^'s for (3 G 
Q| n Q|, the coefficients of L^j and L^, for /3 G Q'| n Q| n T^cx being either 
conjugate or anticonjugate according to the sign in the equality Z^ = ^Zp. 

7.2. Semidefinite Levi forms. 

Lemma 7.1. Let TZ he a root system. There exist no triples a,P,j G TZ 
with 

j a + a, (3 + p, -f + ^ en, 

|a + Q;//? + /3, a + ;9 = 7 + 7. 

Proof. Note that a, (3, 7 belong to the same irreducible component of TZ. 
Thus we may as well assume that TZ is irreducible. We argue by contradic- 
tion. 

The root subsystem TZ' generated by {a + a, /? + ;5, 7 + 7} is of type B2. 
Indeed, it is not of type G2 because it is a proper subsystem of a larger root 
system, since all roots in 7^' are real, while the fact that a + a is a root 
implies that TZc^ is not empty. Moreover, from {a + a) + {P + P) = 2(7 + 7), 
we deduce that TZ' has rank 2 and contains roots of different lengths. Thus 
we can choose a basis {ej} in [)j^ such that 

a + a = 2ei, /3-|-;5 = 2e2, 7 + 7 = ei + e2, 

and, furthermore, that 

a = ei + aes, /? = e2 + aes, 7 = (l/2)ei + (l/2)e2 + be3 + ce4 

for some a,b,c £ M, and a > 0. 

Let {a\P) = 2{a\l3)/{a\a). Recall that, if 01,02 G 7^ are not proportional, 
then ((ai|o2)(o2|ai)) G {0, 1, 2, 3}. Since 

2 

(o|7 + 7)(7 + 7|a) = r—^ — 2' 
1 + a^ 

we have a = 1, that is o = ei + es and /? = e2 + es. Then 

(7I0-I- a){a -|-a|7) 



^1/2) + 62 + c2 

implies that b^ + c^ = 1/2. Hence II7IP = 1. This gives a contradiction, 
because ||a|p = 2 and ||a + op = 4, and at most two different root lengths 
are allowed in an irreducible root system. D 



As a consequence of Proposition 6.3 we obtain 



Proposition 7.2. Let Af (g, q$) be a G-orbit in the complex flag manifold 
F, and'^{M) the distribution of its (0,1) -vector fields. Set 

(7.15) /C$ = {aG7^|0Sc6q^}. 
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Then 
(7.16) 

and we have 
(7.17) 



/C$ 



aG Q 



either —(a + a) ^ Q') 
or ha+a is indefinite) 



'-q* 



D' 



E 



0a- 



^ sufficient condition, in order that ^{M) satisfy the higher Levi form 



concavity condition (|2.32|), is that 
(7.18) 



for each 7 E QJ with 7 = 7 and L^ > 0, 

there exist ao S Q$, and ai, . . . , q,. G /C$ U IC^ s.t. 



,Ei=o aj£nforl<h< r, -7 = ^^^q 



a-;. 



Proof. Denote by Ti® the set of real roots 7 G Q^ n Q^ such that the 
corresponding Levi form L^ is semidefinite. 

Consider any Levi form L = X]/3eQ"nQ" C/3L/3, with c^ = ic/j. We claim 
that if L is semidefinite then 

(I kerL^ C kerL. 



This claim implies (7.16) and (7.17), from which the last statement follows. 



With respect to the basis {Za}aeQ\Q^ ^^^ complexified Levi forms L/3 have 
mutually non overlapping nonzero entries, and each of them has at most one 
nonzero entry in each row (or column). Note that, for /5 G Q^nQ^PlT^cx, the 
matrix of L^ has no diagonal entries. Partition QJ n QJ into the following 
subsets: 

"^0 = {/? G Q% n Q$ n T^rc I L^ is semidefinite (hence diagonal)} , 
Wi = {/? G Q| n Q$ n TZrc I L/3 is indefinite and diagonal} , 
7^2 = {/? G Q% n QJ I L^ has no nonzero entry on the diagonal} , 

is indefinite and has nonzero 1 



W3 = <^ /? G Q| n Q| n 7^r, 



entries on and off the diagonal [ 



If a coefficient cp in the decomposition of L is different from zero for some 
/3 G Wi, then L is indefinite. The same is true for (3 G TLz- Indeed, /3 is real 
and there exists 7 such that /3 = 7 + 7 because L^ has a nonzero diagonal 
entry. Moreover, there are roots q, a' G Q such that (3 = a + a' because L^ 
has a nonzero entry off of the main diagonal. By Lemma [7. 1[ the restriction 
of L^ to 0°^ + 0° is indefinite. Hence the only possible nonzero coefficients 
are those corresponding to roots in TIq = TC® and 7^2- The statement then 
follows from the following statement about Hermitian matrices: 

Let A = (ajj)i<jj<£ be a Hermitian symmetric matrix with ai^i = for 
i = 1, . . . ,i, and D = diag(di^i, . . . , di/). If D + A is semidefinite then 
ker D dkeiD + A. U 



By using Proposition 3.8 we obtain 



Proposition 7.3. Let q he a semisimple real Lie 
algebra of q and B a system of simple roots of TZ^ 



"jra, f) a Cartan suh- 
Let^ C^CB. 
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If '5{M{g,q^)) is subelliptic, then '^{M{q,c\^)) is also subelliptic. D 

Proof. The inclusions q,^ C q<i- and Q$ C Q<si induce a smooth submersion 
M(q,q$) — > M(q,q,j,), that is also a CR map and satisfies the hypotheses 
of Proposition |3.8[ D 



7.3. Minimal orbits. In [24i it is shown that there is a unique G orbit in 
F (the minimal orbit) that is compact. It is connected and has minimal 
dimension. Minimal orbits have been studied, from the point of view of CR 
geometry, in [Ij. If (g, q) is the CR algebra of a minimal orbit M, then go 
contains a maximally vectorial Cartan subalgebra f) of g. This choice of f) 
is equivalent to the fact that TZim = TZ,, i.e. that all imaginary roots are 
compact (see [3]). 

The fact that M = M(g, q) is minimal is then equivalent to the possibility 
of choosing the positive root system T^"*" C Q in such a way that 

(7.19) for a e 7^cx a >- ^^ a>-0. 

This leads to a complete classification of the minimal orbits in terms of 
cross-marked Satake diagrams (see e.g. [1]), i.e. by systems ^ C B, where 



B are the simple roots of a positive system 7?."'" satisfying (7.19). Using 
these $'s, we give below a complete classification of the minimal orbits for 
which the distribution of (0, 1) vector fields satisfies the higher order Levi 



concavity condition (2.32). 



When g decomposes into a sum g = g^^^®- ■ -©g*-^-* of simple ideals, the CR 
manifold M(g,q) is a Cartesian product M(g(^\q(^)) x ••• x M(g(^), q(^)), 
(see [H p.490]). Thus we can reduce the question of the validity of the 
higher order Levi form concavity condition, and also of the subellipticity 
and hypoellipticity of 3(Af(g,q)), to the case where g is a simple real Lie 
algebra. Namely, this property will be valid for M{q, q) if, and only if, it is 
vahd for each factor M(g(''), q^'^)), tor 1 < h < £. 

Thus we state the following classification theorem for the case of a sim- 
ple g. 

For the Satake diagrams characterizing the different simple real Lie alge- 
bras, their labels and those of the roots of a simple positive system, we refer 
to [To], or to the Appendix in [Ij. 

Theorem 7.4. Let (g, q$) be the CR algebra of a minimal orbit M , with 
g simple and assume that M is a CR manifold of finite type in the sense 
of |1]. Then 'b{M) satisfies the higher order Levi form concavity condition 



( |2.32 ) if and only if one of the following conditions is satisfied 

(1) g is either of the complex type, or compact, or of one of the real types 
AE, AHb, B, CEb, DI, DI, DMa, EI, EIV, EVI, EVE, EK; 

(2) g ~ su(p, q) is of type AEIa — PJsf and <I> n {op, Qg} = 0; 

(3) g ~ sp(p,^ — p), with 2p < £, is of type CEa and either ^ is all 
contained in {02^-1 | 1 < i < p} U {oj \ 2p < j < i}, or ^ is all 
contained in {a2j-i | 1 < J < p}; 

(4) g ~ 50*{2i), with i G 2Z+, $ n {ae-i,ai} = 0; 

(5) g is of type EEI and ^ C {03, 04, as} = TZ» n B; 

(6) g is of type FI and ^ C {01,02}- 
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Note that the condition for M being of finite type in the sense of jl] is 
exphcitly described in terms of $ in [H Theorem 9.1]. 

Proof. We use the results of |lj §13, §14]. We stress the fact that we are 
assuming that M is of finite type, so that some choices of ^ are excluded 
from our consideration because of [U Theorem 9.1]. 



First we prove that if (2.32) holds true, then (0,c|#) satisfies one of the 
conditions (1) . . • (6). If (g, q$) is not one of those listed in the statement, 
in all cases, with the two exceptions of CEa with a2p-i G ^ and ^ n {ah \ 
h > 2p} 7^ 0, and FE with 03 € <I>, then M admits a G-equivariant fibration 
onto one of the manifolds described in [Ij Examples 14.1, 14.2, and 14.3]. 
These are strictly pseudoconvex, hence M does not satisfy the higher order 
Levi form concavity condition by Lemma |5.4[ The two remaining cases will 



be discussed below, while proving the opposite implications. 

We know from |T], Theorem 13.4] that M is essentially pseudoconcave, 
and hence in particular (7.18) and ( |2.32 ) hold, in case one of the following 
is satisfied: 

(1') case (1), 

(2') case (2) with either $ C 7^„ or $ n 7^, = 0, 

(3') case (3) with either $ C {ai, . . . , a2p-i}, or <I> C {a2p+i, ■ ■ ■ , ai}, 

(4') case (4), 

(5') case (5) when either 04 G $ C TZ,, or $ = {03, 05}; 

(6') case (6) when $ C {ai,a2}- 
In all of these situations the subellipticity of 3{M) was already proved in 

m- 

To complete the proof, we need only consider the cases in the list in 
which M is not essentially pseudoconcave. Next we proceed to a case by 
case discussion. 

Alia. In this case q ~ 5u{p,q) with 2 < p < q. Let ^ = {aj^, . . . ,ajj.}. 
We need to discuss the case where $ n {op, Oq} = 0, but $ intersects both 
7^, and its complement B \ TZ,. By the condition that M is of finite type, 
we know from [I] Theorem 13.4] that $ does not contain at the same time 
a simple root Oj, with 1 < j < p, and its symmetrical Op+q-j. Since $ 
and ^' = {ap^q-j \ aj G ^} define anti-isomorphic CR manifolds, we can 
assume in the proof that 

l<ji<---<ja<p< ja+i < ■■■ < jb <q < jb+i < ■■■ <jk < jk+i =p + q, 

p- ja < Jb+1 - q- 

The real roots that correspond to non zero semidefinite Levi forms are 

p+q—s 

7^ = ^ aj for p + q- jb+i < s < ja- 

j=s 

Indeed, those a G 2$ \ Q$ for which g^ is not contained in the kernel of 
the Levi form L-^ must satisfy supp(a) n <I> = supp(7) n ^. 

Thus all roots a in Q$ \ Q$ with supp(a) Ci ^ ^ {aj^^aj,^} belong to 
/C$. This is the case in particular for all the simple roots not in Q and for 
the roots —Yl'j=p'^j ^^"^ ~ S?=p+i'^ii whose conjugates are — Og and —Op, 
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respectively. This implies that all roots in — (QJ n QJ) are sums of roots in 



/C$ U/C$, giving condition (7.18). 



CHa. We need to consider the cases where ^ = {a^^, . . . ,aj^.} contains at 
the same time a root Oj with j < 2p and a root Oj with j > 2p. This means 
that k > 2, and we can order the indices in such a way that 

1 < • • • < Ja < 2p < ja+i <■■■<£, 

with jr odd for 1 < r < a. The positive real roots are 

e-i 
Jr = a2r-i + ai + 2 y^«j. 

j=2r 

Since their supports contain aj„+i, they all belong to QJ n Q^. Let ja = 
2q — 1. The Levi forms L^^ are indefinite if r > g by [1;, {ii), p. 520]. Next we 
note that L^^. with 1 < r < q are identically zero, because there is no root 
a in Q$ \ Q$ with supp(a) n $ = supp(7r) n $. Indeed, if a is a negative 
root whose support contains supp(7r) n ^ for some 1 < r < q, then a2q-2 
and a2q both belong to supp(a) and hence also to supp(a). This implies 
that also a2q-i G ^ belongs to the support of both a and a, showing that 
-a G Q| n Q|. 

It was shown in [U p. 520, (iii)] that h^^ is not zero and is positive semi- 
definite. The set of pairs /3, [3' £ 2$ \ Q$ with /? + /3' = 7<j was shown in [H 
p. 520, {in)] to consist of the pairs (/3s, /?s) with 

s 
Ps = - X^Oj for 2p < S < ja+i. 
i=2<? 
We now distinguish two cases, li q < p, the root —a2p belongs to /C$. Then 
the root system generated by /C$ U /C$ contains — (Q$ n QJ) and hence the 



higher order Levi form concavity condition (7.18) is satisfied. 



When p = q, then no element of (/C$ U /C$) \ TZ~^ contains a2p in its 



support. Thus condition (7.18) fails for 7^ because, in the expression for 
(— 7g) in (7.3), the coefficient of a2p is (—2), while for all roots in Q$ it is 
>(-!)• 

Em. We only need to consider the cases where either $ = {03}, or <I> = {05}. 
Due to the symmetry of the EIE diagram, we can restrict our attention to 
the case where $ = {03}. 

We note that 7 = ai + 2a2 + 2as + 804 + 205 + oq is the unique positive 
real root. It belongs to Q^ n Q^ and the corresponding Levi form L^ is 
semidefinite, having rank 1. Indeed, a = —ai — 02 — 2as — 204 — 05 is 
the unique root in Q$ \ Q$ for which a + a = —7, and there is no other 
pair P, j3' G 2$ \ Q$ for which 13 + P' = —7. Thus all simple roots belong 



to /C$ U /C$, and hence the higher Levi form concavity condition (7.18) is 
satisfied. 

FI. We are left to discuss the case where {03} C $ C {ai, 02, 03}. There is 
only one positive real root, namely 7 = ai + 2a2 + 803 + 2a4, that belongs 
to Q^ n Q^. The corresponding Levi form L^ has rank one, and hence it 
is semidefinite. The set KL^ is the complement of {—04} in Q$. Then no 
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element of (/C$U/C$) \7^^ contains a4 in its support. Thus condition (7.181 



fails for 7 because, in the expression for (—7) in (7.3), the coefficient of 04 



is (—2), while for all roots in Q$ it is > (—1). D 
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